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This applied research methods report is a guide for state and local education agency 


policymakers and their analysts who are interested in studying teacher mobility and 
retention. This report is the second in a two-part set and builds on the foundational 
information in report 1. This report presents guidance on how to interpret differences 
in mobility and retention rates by teacher, school, or district characteristics; analyze 
year-to-year trends in mobility and retention; compare mobility and retention rates 
across districts or states; and examine how the implementation of a policy related to 
teachers might be associated with teacher mobility or retention. 
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WHY THIS REPORT? 


This report presents guidance on how to examine teacher 
mobility and retention rates to address a range of objec- 
tives and how to support stakeholders in interpreting and 
using the information for policy decisions. Specifically, this 
report covers: 


e Interpreting differences in mobility and retention rates 
by teacher, school, or district characteristics. 


e Analyzing year-to-year trends in mobility and retention. 


¢ Comparing mobility and retention across districts or 
states. 


e Examining how the implementation of a state or district 
policy related to teachers might be associated with 
teacher mobility or retention. 


This is the second of a two-report set 
ror-) | =XoW-Valoli4lale mm eele glam Vee) ilimvVael sre) 
Retention: Guidance and Considerations. It 
builds on the first report, which provides 
state and local education agency policy- 
raate) <=1acw-] ale Meld al =) mt =10] 0 (er-] ule) ams} m=) <-) ale) (o(-1 as) 
MUivameal=mcolelater-inlolar-)mlaixe)aaat-laleamat-t-vol=re| 
i Ko Jal-) \74om Kr-le1 al =lamaarele)iiinvarclalemacinclala(elan 


Before reading this report, readers might 
find it useful to read report 1. Those who 
Flacwmir-laallitelandidamer=| (0) tial atemaavele)|liavar-lare| 
retention rates might opt to begin with 
this report, which provides more detailed 
guidance on answering teacher mobility 
and retention research questions. 


State and local education agency data analysts can use this report as a guide for undertaking 
analyses that meet these objectives. In turn, the analysts can work with policymakers in 
interpreting and using the results for policy decisions. See box 1 for definitions of key terms 


used in the report. 


This report explains the process of conducting the above analyses in a manner easily acces- 
sible to nonanalysts. For each type of analysis, there are a description of what to consider, 
hypothetical examples, and a box with a more detailed sample analysis using information 
from one or more of the three scenarios introduced in report 1 and described in box 2 
below. The three scenarios are realistic hypothetical examples informed by more than two 
dozen studies on teacher mobility and retention. The hypothetical information in these sce- 
narios was drawn from Regional Educational Laboratory publications and other widely cited, 
peer-reviewed research journals from the past 15 years.' In all these studies the key outcome 
is mobility or retention, and the focus is teachers rather than administrators. 


The guidance in this report is not solely for data analysts but for a multidisciplinary stake- 
holder team of policymakers, program managers, data managers, and data analysts. Indi- 
viduals without a background in research or data analysis might find this report useful for 
improving communication with data managers or analysts. The report might also improve 
understanding of analysis decisions and their implications—policy and other—on the results. 
However, carrying out several of the steps will require someone with data analysis experi- 
ence. So it is advisable to work in teams and involve staff with data analysis experience. 


1. Appendix B in report 1 presents a matrix of the studies that informed the report set, indicating the data 
elements, such as educator and school-level variables, used in each report. Appendix D in report 1 provides a 


summary of each study. 
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Box 1. Key terms 


The following definitions of key terms are presented as a quick reference only. There are variety of ways 
in which these simple definitions might not be sufficient—for example, what to call teachers who change 
roles or take a temporary leave. For a thorough discussion of the nuances of these terms, see report 1. 

e Attrition rate. The percentage of teachers who are leavers. 


Leavers. Teachers who left their initial administrative unit in a given timeframe and did not enter 
another administrative unit during that timeframe. 


Mobility rate. The percentage of teachers who are movers. 


Movers. Teachers who moved from their initial administrative unit to a different administrative unit 
ina given timeframe. The administrative unit that determines whether a teacher is a mover might 
differ from the one used to determine whether a teacher is a leaver. For example, a mover might be 
defined as a teacher who moves between schools within a state, whereas a leaver might be defined as 
a teacher who is no longer teaching in the state. 


Retention rate. The percentage of teachers who are stayers. 


Stayers. Teachers who remained teaching in at least one of their initial administrative units in a given 
timeframe. For example, a teacher who splits time between two schools might be defined as a stayer if 
the teacher remains in one of the schools. 


Trend plot. A visual display of data over time. Also called a line graph. Each data point is plotted ona 
chart where the horizontal axis represents time and the vertical axis represents frequency counts or 
percentages. 


Box 2. Example scenarios and additional research questions 


The following three scenarios represent common analyses that state and local departments of education 
might undertake. These scenarios were introduced in report 1, which detailed how mobility and retention 
rates could be calculated to answer research questions. In the current report these same scenarios are 
used to illustrate how the answer to those original research questions often leads to additional research 
questions and how State and local education agency data can approach solving those new research ques- 
tions. This process could continue iteratively, such that the answer to a new set of research questions 
leads to yet another set of research questions. 


Scenario 1: Mobility and retention among beginning teachers 

Stakeholders from a small state (State A) want to understand the extent to which beginning teachers who 
completed a teacher preparation program in the state are moving between schools, staying in their initial 
school, and leaving teaching positions in the state public school system. The stakeholders articulated their 
original research question as: 


e What percentages of beginning teachers prepared in State A move to another school, stay in their 
school, and leave teaching in State A public schools? 


The analyst calculated retention and mobility rates for State A following the process outlined in report 1 
and then held a meeting to share the results with the stakeholders. During the meeting the state stake- 
holders express interest in examining the patterns of mobility and retention by teacher and school charac- 
teristics, such as whether teachers with certain certifications or in high-need schools are leaving at higher 
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rates. The state stakeholders are also interested in whether differences exist between beginning teachers 
prepared in State A and beginning teachers prepared out-of-state. The state stakeholders also express 
interest in whether a relationship exists between teacher participation in the state’s mentoring program 
and retention, because they are aware that some teachers participate with more fidelity than others. 

At the end of the meeting, the analyst and the stakeholders agree on the following two new research 
questions: 


¢ Do patterns of mobility, retention, and attrition rates among beginning teachers vary by teacher char- 
acteristics (such as certification type or in-state versus out-of-state preparation) and school character- 
istics (such as high- or non-high-needs schools)? 

e Is there a relationship between retention of beginning teachers and the fidelity with which they partici- 
pated in the state’s mentoring program? 


Scenario 2: Mobility and retention in high-need schools 

Board of education members in a large urban district in State B have heard anecdotal reports that 
teachers are leaving high-need schools at higher rates than they are leaving more affluent schools in the 
district. The board members asked the district’s data analyst to investigate the issue. The stakeholders 
articulated their original research questions as: 


e What are the teacher mobility and retention rates in District 1 for the past five years? 
e How do the rates compare between teachers in high-need and teachers in non-high-need schools? 


The analyst calculated the three-year retention rate for each school in District 1 for the past five years as 
well as the in-district retention and mobility rates for the entire district for the past five years. The analyst 
communicates these findings, which pertain to the first research question, to the board members in the 
interim report and confirms that the board is still interested in comparing high-need and non-high-need 
schools, the second research question. 


Scenario 3: Relationship between the institution where a teacher is prepared and teacher retention 
Postsecondary and secondary school policymakers in a populous state (State C) are collaborating to 
better understand how teachers’ preservice experience is related to their retention. The policymakers are 
particularly interested in whether teachers from the state’s public institutions of higher education have 
higher retention rates than teachers from private institutions. The policymakers articulated their original 
research question as: 


e ls there a relationship between the type of institution (public or private) where a teacher is prepared in 
State C and that teacher’s likelihood of staying in or leaving the State C public school system? 


After conducting an analysis to answer this question, the analyst informed the policymakers that teachers 
prepared in private institutions have higher retention rates. After hearing this result, the policymakers 
ask the analyst why that might be the case. The analyst and policymakers then generate a new research 
question: 


e What factors might explain the observed difference in retention between teachers prepared in public 
institutions and teachers prepared in private institutions? 


ANALYSIS STEPS 


Report 1 addressed the initial steps for conducting teacher mobility and retention research. 
Those steps include stating research questions, determining administrative units and time- 
frames, defining mobility and retention, determining data requirements, preparing data, and 
calculating rates. 


Report 2 presents four additional steps that state or local education agency staff members 
can undertake to further examine the mobility and retention rates calculated in report 1. 
Following these steps will yield information for making evidence-based decisions: 


e Step 1: Determine analysis method and data needed to address additional research 
questions. 


e Step 2: Prepare data for analysis. 
e Step 3: Produce frequencies and other descriptive statistics. 
e Step 4: Examine relationships using regression models. 


Although a data analyst will typically perform steps 1—4, getting input from individuals with 
contextual knowledge of the policies in place that influence teacher mobility and retention 
can be beneficial. A collaborative approach helps ensure that the data analyst investigates 

the most useful research questions and that the results are accurate. The results should be 
discussed with the multidisciplinary team of staff members to interpret their meaning and 

discern their implications. 


Step 1. Determine analysis method 
and data needed to address additional 
research questions 


Step 1 focuses on deciding how to address additional research questions on mobility and 
retention rates that emerge from initial results. This process begins where step 6 in report 1 
leaves off—after stakeholders have basic mobility and retention rates and have discussed 
the implications of the results. That discussion is likely to lead to new or updated research 
questions (see box 2) for which the data analyst will determine whether additional data are 
needed. For example, if stakeholders want to examine whether mobility and retention rates 
vary by certain teacher and school characteristics, the analyst will need to check whether 
those characteristics are included in the data files used to calculate the mobility and reten- 
tion rates. If they are not, the analyst will need to add those variables to the data files, fol- 
lowing the procedures outlined in step 5 in report 1. 


Next, the analyst will select an appropriate analysis method for addressing the research 
questions, often choosing between a simple descriptive summary statistical analysis and 


Analysis steps 


an advanced statistical analysis. Questions that concern simple differences in mobility and 
retention rates by school characteristics can be examined descriptively by simply disaggre- 
gating the rates by the school characteristic categories, such as socioeconomic status and 
locale. Other questions might require more advanced statistical analyses, such as regression 
analyses, to identify associations between variables such as teacher mobility rates and 
school locale. An in-depth discussion of situations that might require regression analyses is 
in step 4. 


The choice of analysis method is a complex matter, and it is common for analysts to rely on 
multiple analysis methods to answer a research question. For example, “Are teachers in rural 
districts more likely to leave the district than teachers in urban districts?” could be answered 
in multiple ways. 


One analyst might employ simple descriptive summary statistics at the district level, using 
district retention rates and calculating the mean retention rate separately for rural and 

for urban districts. The analyst might calculate simple means or weighted means using 
district enrollment. Whereas simple means would give equal weight to a small district and 

a large district, weighting by enrollment would give more weight to a large district. The 
choice between the two should be guided by both the research question and the discussion 
between the analyst and other stakeholders. Simple means are easy to calculate and com- 
municate but are susceptible to biases from extreme values. For example, smaller districts 
are more likely than larger districts to have extreme mobility and retention rates due to their 
smaller sample sizes.’ If a concern exists regarding small districts biasing the results for the 
entire state, the analyst might advise stakeholders of the merit of weighted means, despite 
their complexities. 


Another analyst, however, might take a different approach. Instead of calculating a dis- 
trict-level retention rate, the analyst might use teacher-level frequency data on the number 
of leavers and stayers for each district. The analyst might then total the number of leavers 
and stayers separately for urban and rural districts, while keeping the administrative unit 

as the individual district. These totals could be summarized in a two-by-two cross-tabula- 
tion, where the rows are “urban” and “rural” and the columns are “stayers” and “leavers” 
(table 1). The analyst might add a statistical test of association, such as a chi-square test, to 
this descriptive summary to see whether the relative frequency of stayers and leavers differs 
significantly between urban and rural districts. 


Table 1. Example cross-tabulation of stayers and leavers by district locale 


[Wofer-) (-) Stayers Leavers 
Urban district Total count of stayers in the urban district Total count of leavers in the urban district 
Rural district Total count of stayers in the rural district Total count of leavers in the rural district 


Source: Authors’ creation. 


2. For example, a small district with only 1 beginning teacher will have a retention rate of 100 percent or 
O percent, depending on the movement of this single teacher, whereas a large district with 50 beginning 
teachers will rarely have such an extreme retention rate. 
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The choice of analysis method often reflects the data available and the analyst’s personal 
preferences. In the above example the second approach would not be feasible if the analyst 
had access to data on district retention rates but not to data on the frequencies of stayers 
and leavers. 


As these examples illustrate, there is no clear-cut way to determine the most effective analy- 
sis method. In fact, the question in the example could also be analyzed using more sophis- 
ticated approaches, including linear regression, logistic regression, and survival analysis. 

But some analysis methods are more appropriate than others, depending on the type and 
nature of available data. Nonresearchers such as policymakers and program managers have 
important roles in advising data analysts, by communicating what outcomes (dependent 
variables) and predictors (independent variables) they are interested in. Data managers’ role 
is then to inform the analysts which outcomes and predictors are available and what their 
quality, completeness (missingness), and form (for example, categorical, ordinal, or interval) 
are. The analyst will take those pieces of information, along with additional considerations 
such as whether the data are from matched groups or reflect independent groups, to decide 
on a proper analysis method. Several existing resources describe these terms and their dif- 
ferences and how to choose an appropriate analysis method.? 


One important consideration when determining the best analysis method is whether simple 
descriptive summary statistics will suffice as opposed to sophisticated, and potentially com- 
plicated, advanced statistics. In general, advanced statistics, such as regression analyses and 
other inferential statistical methods, are better suited when the research question aims to 
uncover a complex relationship in the data. However, advanced statistics can be difficult to 
interpret and explain to a general audience, which could cause unnecessary confusion and 
delay in policy decisionmaking. 


One approach is to perform a simple analysis whose results are communicated to a wide 
audience and to perform a more sophisticated alternative to verify that claim. If the more 
sophisticated analysis corroborates the results of the simpler analysis, the simpler analysis 
could be used for presentations or reports, and the more sophisticated analysis could be 
placed in an appendix as a technical reference. In some cases the purpose of the analysis 
demands the more sophisticated alternative. For example, regression analysis is needed 

to predict retention based on teacher or school characteristics. But if the goal is simply to 
describe the relationship between retention and a few teacher or school characteristics, a 
descriptive summary, such as cross-tabulations with a chi-square test of association, would 
suffice (box 3). 


Once the analysis method has been chosen and additional data needs have been identified, 
the analyst prepares the data file, following the considerations discussed in report 1. 


3. For example, Loeb et al. (2017, available at https://ies.ed.gov/ncee/pubs/20174.023/pdf/20174.023.pdf) and 
the Institute for Digital Research and Education’s Statistical Consulting site (https://stats.idre.ucla.edu/other/ 
mult-pkg/whatstat/). 
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Box 3. Analysis methods for the example scenarios 


This box describes options for analysis methods for each example scenario. Detailed discussions of the analysis 
methods and visual displays are presented later in the report. 


Scenario 1: Mobility and retention among beginning teachers 

State A stakeholders want to understand the extent to which beginning teachers in the state are moving between 
schools, staying in their initial school, and leaving the profession and whether the results show patterns by 
teacher and school characteristics. They also want to know whether there is a relationship between teacher par- 
ticipation in the state’s mentoring program and retention, because they are aware that some teachers participate 
with more fidelity than others. 

To answer the first question, the analyst examines mobility and retention disaggregated by teacher and 
school characteristics. Using cross-tabulations, the analyst produces frequencies of the number of movers, 
stayers, and leavers by teacher and school characteristics. Then, to compare mobility, retention and attrition 
rates across groups, the analyst computes means for each subgroup and summarizes them in bar graphs. Both 
cross-tabulations and comparisons of means permit simple tests of statistical significance: a chi-square test of 
association for cross-tabulations and a t-test for comparisons of means. 

To answer the second question about relationships, the analyst uses regression analysis. Because the 
outcome variable is binary (retained or not retained), logistic regression is performed. Using a logistic regression 
model enables the stakeholders to understand the relationship between an individual teacher’s fidelity of imple- 
mentation and that teacher’s likelihood of retention, after other factors are controlled for. 


Scenario 2: Mobility and retention in high-need schools 
Board of education members in District 1 in State B have heard anecdotal reports that teachers are leaving high- 
need schools at higher rates than they are leaving more affluent schools in the district. The board members have 
asked the district’s data analyst to examine annual teacher mobility and retention rates for the past five years and 
to compare the trends in those rates among teachers in high-need schools and among teachers in non-high-need 
schools. 

The analyst performs a visual examination of five-year trends, disaggregated by high-need schools and non- 
high-need schools. The analyst creates a line chart with two trend plots, one representing the mean retention 
rate for high-need schools and the other representing the mean retention rate for non-high-need schools. 


Scenario 3: Relationship between the institution where a teacher is prepared and teacher retention 
Postsecondary and secondary school policymakers in State C are collaborating to better understand how teach- 
ers’ preservice experience is related to their retention. The policymakers are particularly interested in whether 
teachers from the state’s public institutions of higher education have higher retention rates than teachers from 
private institutions. 

The analyst examines retention disaggregated by teacher characteristics. Because this concerns individual 
teachers’ characteristics, the analyst analyzes frequency of cases using cross-tabulations. The analyst then follows 
this up with a chi-square test of association. 

The analyst then reports back to the policymakers that teachers prepared in private institutions have higher 
retention rates. This leads the policymakers to ask why. The analyst wonders whether the observed difference 
might disappear when other factors are taken into consideration—in other words, when other factors are sta- 
tistically controlled for. The analyst concludes that the appropriate analytic method is a logistic regression with 
covariates. 


Analysis steps 


Step 2. Prepare data for analysis 


Step 2 involves preparing the data file for analysis and expanding it as needed. If teacher, 
school, or district characteristics were not part of the first file created but are needed so 
that the research questions can be addressed, they will be merged into the data file in this 
step. For a review of how to merge files, see step 5 in report 1. 


Ensuring consistency 


When an analytic data file is compiled by merging mobility and retention data files for differ- 
ent years or different states, it is important to check for consistency in the definitions of the 
mobility and retention variables and in the coding structure for those variables. Answering 
research questions using mobility and retention rates, which were calculated through the 
process outlined in report 1, often involves combining mobility and retention data for differ- 
ent years or different states. That makes ensuring consistency especially important. As 
report 1 discusses in detail, there are multiple ways in which these terms can be defined and 
thus calculated. If the data files to be merged are not consistent in their definitions or calcu- 
lations of mobility and retention variables, the analyst is advised to decide on the preferred 
or most relevant definition and calculation and to apply it to each file prior to merging. The 
analyst might need to consult with the multidisciplinary team to decide on the preferred 
definition and calculation to apply across files. 


Step 3. Produce frequencies and other 
lit, descriptive statistics 
i 


Step 3 describes how to produce frequencies and other descriptive statistics for three 
common purposes: to examine differences in mobility and retention rates by teacher, 
school, or district characteristics; to create visual displays of year-to-year trends in mobility 
and retention; and to compare mobility and retention rates across districts or states. 


Examining differences in mobility and retention rates by 
teacher, school, or district characteristics 


There are two primary approaches to help stakeholders understand differences in mobility 
and retention rates that are disaggregated by teacher, school, and district characteristics: 
calculating means to compare rates and producing cross-tabulations to compare frequen- 
cies. For the comparison of mean mobility and retention rates, the mean is calculated for 
each subgroup defined by the characteristics. For example, if the research question asks 
about retention rates by schools in large versus small districts, in place of the overall mean 
for the three-year retention rate of beginning teachers, the analyst would calculate reten- 
tion rates separately for schools in large districts and for schools in small districts. The result 
of that analysis could be summarized as a table or as bar graphs for visual analysis, as in 
figure 1. 
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Figure 1. The mean three-year retention rate was higher for small districts than for large 
districts, 2012/13-2014/15 


@ Small districts (n = 350) ® Large districts (n = 700) 
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Source: Hypothetical data used for illustration purposes. 


When the disaggregation is conducted using frequency data—in other words, on the 
number of stayers, movers, and leavers—a common approach is to use cross-tabulation. 

In cross-tabulations each row represents a mutually exclusive category of teachers—for 
example, stayer, mover, and leaver. Each column represents a mutually exclusive subgroup 
based on the characteristic of interest—for example, teachers from large districts and teach- 
ers from small districts. The cells at each intersection of rows and columns contain the count 
or frequency of cases, which is teachers in this example (table 2). 


Table 2. Three-year teacher retention and attrition rates by district size, 2012/13-2014/15 


Large districts Small districts 
Number Percent Number Percent Row total 
Stayers (retention) 2,050 84 4,000 88 6,050 
Leavers (attrition) 400 16 550 12 950 
Column total 2,450 100 4,550 100 7,000 


Source: Hypothetical data used for illustration purposes. 


Box 4 shows how data are disaggregated by various characteristics in example scenario 3. 
Box 5 discusses the process of conducting a visual analysis of trends in mobility and reten- 
tion in example scenario 2. 
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Box 4. Disaggregating by teacher characteristics in example scenario 3 


Postsecondary and secondary school policymakers in State C are interested in whether teachers from 
the state’s public institutions of higher education have higher retention than teachers from private insti- 
tutions. To examine retention disaggregated by the type of institution from which individual teachers 
received their degree, the analyst examines the frequency of cases using cross-tabulations. The analyst 
and the policymakers then agree to use the latest figure for five-year retention in the state public school 
system, which comes from the 2015 cohort of beginning teachers. 

To create a two-by-two cross-tabulation of (stayer, leaver) x (private, public), the analyst classifies all 
teachers in the 2015 cohort into four groups: stayer, graduated from private institution; stayer, graduated 
from public institution; leaver, graduated from private institution; and leaver, graduated from public insti- 
tution. The analyst then enters the number of teachers in each category in the corresponding cell in the 
cross-tabulation and calculates the row, column, and grand totals (see table). Finally, the analyst calculates 
row and column percentages for each cell. After examining the cross-tabulation, the analyst concludes 
that teachers trained in private institutions have higher five-year retention rates than teachers trained in 
public institutions. 


Example cross-tabulation of stayers and leavers by institution type 


Stayers Leavers 
Institution type Number Percent Number Percent Row total 
Private A A/(A+C) Cc C/(A+C) A+C 
Public B B/(B+D) D D/(B+D) B+D 
Column total A+B (A+B) / C+D (C+D) / A+B+C+D 
(A+B+C+D) (A+B+C+D) 
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Box 5. Visual analysis of trends in mobility and retention in example scenario 2 


Board of education members in District 1 in State B and their analyst have defined retention and mobility 
in the city as follows: stayers are teachers and school administrators who are in their original school of 
assignment after three years, movers are teachers and school administrators who are in a different school 
in the district from their original school of assignment after three years, and leavers are teachers and 
school administrators who leave public schools in the district within three years. 

To analyze the five-year trend in school retention, the three-year retention rate for each of the five 
years is calculated for the district as a whole. This results in the trend plot of stayers. To provide context, 
the analyst also adds the trend plot for the entire state. Because the stakeholders are interested in com- 
paring the retention rate of high-need schools with that of non-high-need schools, the analyst adds two 
more trend plots, disaggregating the three-year retention rate for high-need schools and non-high-need 
schools (see figure). 

The analyst follows the same procedure to create the five-year trend plots for the mobility rate in the 
district, which results in the trend plot of movers. The analyst then creates the five-year trend plots for 
the retention rate in the district, which is the sum of stayers and movers. (Alternatively, the analyst could 
count the number of leavers to create five-year trend plots for the district attrition rate.) In addition to 
the three trend plots of retention and mobility rates, the analyst also creates a trend plot for the annual 
frequency of mobility in the district. The analyst adds a trend plot for each of the four types of mobility: 
move from high-need to non-high-need school, move from high-need to high-need school, move from 
non-high-need to non-high-need school, and move from non-high-need to high-need school. This plot is 
used to study the pattern of mobility between high-need and non-high-need schools. 


The three-year teacher retention rate is consistently lower in high-need schools than in non-high-need 


schools 
= District average = High-need schools (n = 20) = Non-high-need schools (n = 30) == State average 
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Source: Hypothetical data used for illustration purposes. 
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Creating visual displays of year-to-year trends in mobility and 
retention 


Trend plots, which provide a visual display of rates over time, facilitate the identification of 
patterns and trends as well as anomalies that require further investigation. Although descrip- 
tive statistics can be summarized in tables, trend plots, also referred to as line graphs, are a 
useful format for visualizing and analyzing patterns in mobility and retention. It is also possible 
to test the significance of the change in mobility or retention between timepoints using a sta- 
tistical analysis method designed for repeated measures. Stakeholders might find trend plots 
useful for forecasting teacher labor force needs. Sometimes trends are analyzed separately 
for different types of teachers, schools, or districts in order to identify where improvement in 
retention is needed, which in turn helps stakeholders prioritize the use of limited resources. 
For example, Lochmiller et al. (2016) examined the annual trend of retention in Kentucky public 
schools from 2008 to 2012 in order to identify a subgroup that needed attention. 


The three-year teacher retention rate in the district in scenario 2 demonstrates the use of a 
trend plot (figure 2). The trend plot includes the disaggregated retention rates for high-need 
schools and non-high-need schools. It also includes the retention rates for the entire state 
for reference. There are several noticeable patterns that the stakeholders and the analyst 
could discuss. For example, for the past five years the average three-year teacher retention 
rate in the state remained relatively stable, except for a dip in 2013/14, which the stakehold- 
ers note was also a year with a state budget crisis. The teacher retention rate is consistently 
lower in high-need schools than in non-high-need schools. And non-high-need schools in 
the district have recovered to their pre-crisis retention rate, as has the state as a whole, but 
high-need schools in the district have not. 


It is important for the analyst to remind stakeholders that visual analysis of trends is just 
a first step in understanding changes in mobility and retention over time. It is useful for 


Figure 2. The three-year teacher retention rate is consistently lower in high-need schools 
than in non-high-need schools 


= State average = High-need schools (n = 250) == Non-high-need schools (n = 750) 
Percent 
90 


= SS 
10 SS  — 


60 


50 
2012/13 2013/14 2014/15 2015/16 2016/17 


Source: Hypothetical data used for illustration purposes. 


12 


Analysis steps 


identifying where needs might exist for maintaining retention, but in and of itself it is insuffi- 
cient for understanding why changes might have occurred. 


Comparing mobility and retention rates across districts or 
states 


Stakeholders are often interested in comparing mobility and retention rates across states; 
however, such comparisons are complicated because rates are not always calculated the 
same way across States or over time. 


Carrying out these comparisons by recalculating the rates so that they are comparable 
requires access to teacher-level data from each state, which is often not feasible. Recalcu- 
lating the statistics to make them comparable is also complicated. For example, if one state 
has a more inclusive definition for stayer than the other, the analyst needs to recalculate the 
retention rates for one of the states so it aligns with the retention rate for the other. After 
such an analysis is conducted, it is important for the analyst to communicate the results to 
the stakeholders of the two states to avoid confusion over the difference between the “offi- 
cial” retention rate and the rate used for the analysis. The issue of compatibility of mobility 
and retention data can also arise for cross-district comparisons. 


Stakeholders in neighboring states might be interested in analyzing teacher interstate mobil- 
ity, such as how many and which types of teachers are leaving one state to teach in another 
state or leaving teaching in public schools in the neighboring states. That could supplement 
information on local supply and demand and help in planning for retention strategies. For 
example, Podgursky et al. (2016) examined teacher retention, mobility, and attrition between 
2005/06 and 2011/12 in lowa, Minnesota, and Wisconsin. That study produced interstate 
mobility rates and thereby addressed a common limitation of single-state studies: limited 
ability to track whether teachers left teaching altogether or moved to teach in another state. 


Merging data across states when teachers might be in the data for more than one state 
requires appropriately matching the teachers in each data file. States do not typically pre- 
serve another state’s teacher ID, so the same teacher could be in the data file with two 
different IDs. To match the teachers, the analyst needs to use other uniquely identifying 
information in combination, such as first and last name, date of birth, or social security 
number. Once the files are merged appropriately, the analyst will need to identify teachers 
as stayers, movers, or leavers. This can be done by identifying each teacher’s beginning-of- 
year primary school, meaning the school at which the teacher spends the most time, and 
then defining teachers who had the same primary school identifiers at the beginning of 
both years being examined as stayers, teachers who had different primary school identifiers 
across the years as movers, and teachers who were not in the last year data as leavers (for 
an example of how to conduct such a comparison, see table 3). 


With the data prepared and stayers, movers, and leavers identified, the analyst can calculate 
teacher retention, mobility, and attrition rates and compare them across states (table 4). 
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Table 3. Example comparison of mobility across different states, 2015/16-2016/17 


Primary Primary 
Yo areXe) M1) Eyal atexe) ML) State State 
Teacher ID 2015/16 2016/17 2015/16 2016/17 Stayer Mover Leaver 
1 A A State 1 State 1 Yes 
2 A B State 1 State 2 Yes 
3 B C State 2 State 2 Yes 
5 D = State 2 = Yes 


— is not available. 


Source: Hypothetical data used for illustration purposes. 


Table 4. Stayers, movers, and leavers, by state, 2015/16—2016/17 


State Stayers Movers Leavers Total 
State 1 

Number 32,000 3,200 4,800 40,000 
Percent 80 8 12 100 
State 2 

Number 45,000 3,500 1,500 50,000 
Percent 90 7 3 100 
State 3 

Number 28,000 3,500 3,500 35,000 
Percent 80 10 10 100 


Source: Hypothetical data used for illustration purposes. 


Step 4. Examine relationships using 
regression models 


Step 4 describes the process of examining relationships using regression analysis, which 

is particularly useful when stakeholders are interested in how multiple teacher, school, or 
district variables are associated with teacher mobility and retention while other character- 
istics are held constant. Regression analysis is also useful when stakeholders are interested 
in whether participation in certain professional development programs is associated with 
teacher retention and mobility after differences that existed before participation are taken 
into consideration. For example, in Espel et al. (2019) state education agencies from Colo- 
rado, Missouri, and South Dakota asked whether certain teacher, school, or district charac- 
teristics were associated with teacher retention. 


Examining the relationship between background 
characteristics and mobility and retention rates 


Regression analysis is useful when stakeholders are interested in assessing the relation- 
ship between mobility and retention and a focal teacher, school, or district variable, while 
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holding other teacher, school, or district variables constant. Consider an example where the 
stakeholders in a state notice that larger districts have lower rates of retention than smaller 
districts and that many of those who leave larger districts are in the early years of their 
teaching career. 


Although discussions of descriptive findings often conclude that the perceived patterns 
represent something meaningful, this determination is problematic because other factors, 
including random events, could produce seemingly meaningful data patterns. For example, 
the lower retention rate among larger districts might be spurious: the pattern might reflect 
only a correlation between the size of the district and its locale in the state. In other words, 
larger districts in the state happen to be predominantly in urban centers with high poverty 
and low academic achievement. These factors might help explain the turnover. 


The point here is that the results of simple descriptive summary statistics often conceal 
alternative hypotheses that are equally or more plausible. So the analyst and stakeholders 
need to exercise caution in interpreting the results of simple descriptive analysis and discuss 
what other factors might be driving the results. 


To account for multiple teacher, school, and district characteristics that can influence mobil- 
ity and retention and to predict mobility and retention rates when investigating a research 
question, many analysts turn to regression analyses. Regression analysis allows analysts to 
examine the potential contribution of certain characteristics while statistically holding the 
potential contributions of other characteristics constant. It also allows analysts to predict 
mobility and retention, using teacher, school, and district characteristics to better plan for 
future hiring. Though these types of analyses are not specific to regression, they lend them- 
selves well to this type of investigation. Regression analysis also differs from descriptive 
statistics in terms of its goal. Specifically, the use of regression is often motivated by the 
analyst’s interest in predicting outcomes. 


It is important to know that the type of variable—categorical versus continuous (see table 5 
for definitions)—does not dictate whether descriptive statistics or regression should be 
used. Descriptive statistics can be used to generate cross-tabulations, for describing associa- 
tions between two or more categorical variables; disaggregated mean tables, for describing 
associations between two or more categorical variables and a continuous variable; and 


Table 5. Definition of key terms for different types of variables 


Categorical variable A variable with two or more values that have no intrinsic order. For example, the binary variable 
retention, with two possible values (stayer or leaver). 


Continuous variable A numerical variable. For example, retention rate. 


Predictor variable A variable or set of variables whose influence on an outcome variable is examined in regression 
analyses. Also referred as independent variable. For example, size of district. 


Outcome variable A variable whose outcome is predicted by a predictor variable or set of predictor variables in 
regression analyses. Also referred as dependent variable. For example, mobility and retention 
rates or the binary variable retention, with two possible values (stayer or leaver). 


Source: Author’s compilation. 
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Pearson’s r correlation coefficient, for describing associations between two continuous vari- 
ables. Likewise, regression analysis can be used to perform linear regressions, for predicting 
a continuous outcome variable with a set of categorical or continuous predictor variables, 
and logistic regressions, for predicting a categorical outcome variable with a set of categori- 
cal or continuous predictor variables. 


An example for the use of regression analysis is to predict district retention rate, which 

is a continuous outcome variable measured on a scale of O to 100. Categorical predictor 
variables in the analysis could include district locale (urban, suburban, or rural) and district 
performance category (low or not low), and continuous predictor variables could include 
total district enrollment, percentage of students in the district who are from racial/ethnic 
minority groups, and percentage of students in the district who are eligible for the national 
school lunch program. An example for the use of logistic regression analysis is to predict 
beginning teachers’ two-year school retention, which is a categorical outcome variable 
(retained or not retained). Categorical predictor variables in the analysis could include the 
presence or absence of mentoring program at the school (mentoring or no mentoring) and 
location of undergraduate institution (in-state or out-of-state), and continuous predictor 
variables could include teacher’s age at the time of employment and the percentage of stu- 
dents in the school who are from racial/ethnic minority groups. 


Regression has a clear advantage over descriptive analysis when the relationship between 
more than two variables is being investigated. Although cross-tabulations and disaggregated 
mean tables can be created in such cases, the number of dimensions could quickly become 
overwhelming. For example, a researcher who wishes to study the relationship among reten- 
tion, school locale, and teacher gender could produce a three-dimensional cross-tabulation, 
resulting in a table with eight cells (2[stayer, leaver] x 2[urban, rural] x 2[male, female]). But 
interpreting patterns in the table might be challenging. As the number of variables increases, 
interpreting data patterns becomes nearly impossible—even for the most experienced ana- 
lysts. On the other hand, an analyst can study the relationship between 5 or even 10 vari- 
ables using regression analysis because it provides readily interpretable results. 


Regression analyses examine the influence of a predictor variable or a set of predictor vari- 
ables on an outcome variable. The equation form of a regression analysis can be written as: 


Mobility (or retention) rate = b, + b,(predictor 1) + b,(predictor 2) + ... + error. 


In this expression the outcome—mobility or retention rate—is predicted by multiple predictors. 
As mentioned, a regression model can handle both continuous and categorical predictors. 
When a categorical predictor consists of more than two values, it needs to be included as a 
series of binary variables. For example, district locale (urban, suburban, rural) would be included 
as b,(urban) + b, (suburban). Typically, one of the values is omitted from a series, because that 
value can be expressed as “No” or “O” in all variables in the series. In this example, the remain- 
ing value of “rural” can be expressed as “No” or “O” in both “urban” and “suburban.” 
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For the example mentioned at the beginning of this section, the apparent pattern between 
the size of the district and retention might be expressed as: 


Retention rate = b, + b,(size of district) + error. 


Analyzing the data with this regression model—called simple regression because there is 
only one predictor variable—might show that the relationship is statistically significant, 
meaning that is unlikely to be a result of random chance. But a subsequent analysis using 
the following regression model—called multiple regression because there is more than one 
predictor variable—might show that the relationship between district size and retention 
rate disappears once the district locale is taken into consideration. The addition of a second 
predictor to the initial regression model to jointly predict the outcome is often motivated 
by the need to estimate the strength of the relationship between the initial predictor and 
the outcome while the influence of the second predictor on the outcome is taken into con- 
sideration. This is commonly referred to as statistical control. In this example the analyst is 
estimating the strength of the relationship between district size and the district retention 
rate while controlling for the locale of the district: 


Retention rate = b, + b,(size of district) + b,(locale) + error. 


Regression analysis can also be used to study the relationship between teacher characteris- 
tics and retention while district characteristics that might also affect retention are controlled 
for (box 6). 


In box 6 the outcome Probability of five-year retention is a continuous variable.’ In some 
cases a regression model with a categorical outcome might be more appropriate to use 
—for which a logistic regression model would be most appropriate. An analyst who was 
interested in examining the relationship between type of preparation program and two 
distinct outcomes, such as teachers who were stayers and teachers who were leavers, 
would need to use a logistic regression in which Retention is a binary variable with two 
possible values (retained and not retained). While useful, logistic regression findings are 
more difficult to interpret. Communicating findings based on logistic regression is often chal- 
lenging for analysts because the results are expressed as the log of odds ratios and lack an 
intuitive meaning. So many analysts prefer converting the results to odds ratios. However, 
stakeholders might also find odds ratios hard to understand. One recommendation is to 
calculate and communicate expected probabilities. For example, the analyst might report 
that the expected probability of a typical new teacher being retained after three years is 

70 percent for urban districts and 75 percent for suburban districts, when district poverty 
level and academic performance are statistically controlled for. Or the analyst might say that 
working in an urban district is associated with a 5 percentage point decrease in three-year 
retention among typical new teachers, when district poverty and academic performance are 


4. The example in box 6 is meant to serve as a simplified model for how to explore retention using regression 
analysis. These models often require additional complexity. Analysts might decide that logistic regression 
or hierarchical linear modeling are the best methods to address their research questions given the available 
data. How to specify the regression model is beyond the scope of this report. Interested readers are advised 
to refer to statistics textbooks such as Howell (2010) or Raudenbush and Bryk (2002). 
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Box 6. Analyzing relationships while controlling for characteristics in example scenario 3 


After examining the difference in the five-year retention rate between teachers prepared in public insti- 
tutions and teachers prepared in private institutions, the policymakers in State C notice that teachers 
trained in private institutions have higher retention rates. But the policymakers are unsure whether the 
mean difference in the retention rate between the two populations of teachers is meaningful or is the 
result of other factors. 

The analyst suggests performing a regression analysis using the model: 


Probability of five-year retention = b, + b,(Type of preparation) + error. 


The analyst asks the policymakers what other characteristics might also be associated with the type of 
preparation program and retention. After some discussion, they consider the possibility that teachers 
who are prepared in private institutions are more likely to take jobs in higher-income districts. Because 
the policymakers are already aware that higher-income districts have higher retention than lower-income 
districts, they ask the analyst to check the possibility that this factor is behind the apparent association 
between the type of preparation program and retention. 


To test this possibility, the analyst performs a regression analysis using the model: 


Probability of five-year retention = b, + b,(Type of preparation) + 
b,(District percent eligible for the national school lunch program) + error. 


After seeing that the significant difference between the two types of preparation programs disappears 
when household income is taken into consideration, the analyst and policymakers decide the finding of 
a private—public difference in retention rate could be better understood as a difference in the type of 
districts in which beginning teachers seek employment. But the analyst cautions the policymakers that 
the results of regression analysis alone cannot be used to draw causal conclusions. Specifically, neither 
type of preparation nor household income level should be considered factors affecting teacher retention 
in the district. Hearing this, the policymakers ask the analyst if there is a way to know whether the type 
of preparation affects retention. In response, the analyst invites the policymakers to discuss the viability 
of conducting studies with rigorous designs that might support causal inference, such as experimental or 
quasi-experimental studies. 
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controlled for. Most statistical packages come with the option of calculating expected proba- 
bilities. For example, the “margins” command in Stata produces such probabilities. 


Examining the relationship between program implementation 
and mobility and retention rates 


Mobility and retention rates are sometimes used as the outcome variable in program evalu- 
ation. For example, a state might be interested in evaluating the efficacy of its new teacher 
mentoring program, whose objectives include improving new teacher retention. If the 
program is introduced to some districts but not to others, the stakeholders and the analyst 
could examine its efficacy by comparing retention rates between districts that participate 

in the program and districts that do not. In addition to randomly assigning districts to the 
program, one method of examining efficacy is to match participating districts and nonpartic- 
ipating districts in terms of retention rate before the program and in terms of other charac- 
teristics that might be associated with retention (such as district academic performance). This 
matching would yield two groups of districts with equivalent pre-program retention rates and 
other characteristics associated with retention and make it unnecessary to hold those factors 
constant during the regression analysis. However, state teacher mentoring programs are often 
universal, in that all districts in the state are required to participate. When that is the case, how 
can the association between program participation and teacher retention be studied? 


One approach is to examine the fidelity of program implementation (the degree to which 
the program was implemented as intended) and its relationship to teacher retention. The 
variations that exist across districts in terms of fidelity of implementation make it possible 
to this. But a positive association between fidelity and retention does not permit causal 
interpretation. Other factors beyond faithful program implementation might be driving the 
higher retention rates. Still, a positive association could lead stakeholders to hypothesize 
that the mentoring program is related to higher retention and that the state might want to 
conduct more rigorous studies to assess the impact of mentoring programs on retention. 


The concept of fidelity of implementation, as well as its measurement, is complex and 
beyond the scope of this report.° But consider an example in which the state mentoring 
program office has monitored the fidelity of implementation in each district and assigned a 
rating ranging from 1, minimal implementation, to 10, full implementation, to each district. 
The analyst could obtain district fidelity ratings and use them in the regression analysis. The 
analyst would start with the following initial regression model: 


Retention rate = b, + b,(Fidelity of implementation) + error 
This regression model® might show that districts’ fidelity in implementing the new teacher 


mentoring program has a weak and nonsignificant positive relationship with the retention 
rate of new teachers. In other words, it might be that the better the fidelity with which a 


5. There are several resources for interested readers—for example, Century et al. (2010) and O’Donnell (2008). 

6. In practice, constructing regression models involves more than choosing predictor variables. For example, 
sometimes variables are transformed using logarithms to give a highly skewed variable a more normal (bell- 
curve shaped) distribution. Other times, an error term is split into two sources, to reflect “nested” structures 
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district implements the new teacher mentoring program, the higher the new teacher reten- 
tion rate for the district. At this point the relationship remains suggestive and therefore 
merits further examination. The analyst might also hear from the program manager that 
affluent suburban districts often have their own mentoring programs that replace the state 
mentoring program, resulting in lower fidelity scores among those districts. The analyst then 
runs separate analyses for affluent suburban districts and the rest of the districts in the state 
and finds a positive and significant relationship between fidelity and retention in the districts 
without their own mentoring program and no relationship in the districts with their own men- 
toring program. The analyst might further refine the regression model by including district 
characteristics that are likely to be associated with teacher retention, such as academic per- 
formance before the introduction of the mentoring program. Regression analysis allows for 
the examination of the relationship between program implementation and retention, and the 
same analysis can be repeated separately for districts with different characteristics (box 7). 


in the data—such as teachers nested in districts. Statistics textbooks such as Howell (2010) or Raudenbush 
and Bryk (2002) provide additional details. 
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Box 7. Analyzing relationships between program implementation and retention in 
example scenario 1 


A primary reason that stakeholders in State A wanted to study retention among beginning teachers is to 
see whether their new statewide teacher mentoring program is improving retention. Eager to know this, 
the stakeholders ask the analyst to verify the effect of the program. The analyst explains that because the 
mentoring program is mandatory to all new teachers in the state, it is not possible to conduct a study that 
draws a causal conclusion, such as an experimental study. Noting the variation across districts in the fidel- 
ity of implementing the program, however, the analyst suggests an analysis of the association between 
the fidelity of implementation and retention. 

The analyst performs a regression analysis using the following model: 


District retention rate = b, + b,(Fidelity of implementation) + error. 


This analysis confirms that districts with higher fidelity of implementation have higher retention rates 
among beginning teachers. As the analyst explains the results, the stakeholders ask if there is any way to 
know whether the strength of the relationship is similar in low-performing districts with historically lower 
retention rates and in other districts with higher retention rates. To answer this question, the analyst 
performs a separate analysis, using the same model described above, for low-performing districts and for 
the rest of the districts in the state. The results confirm that the fidelity—retention relationship is as strong 
in low-performing districts as in the rest of the districts. 

Although the analyst performed two separate analyses, one for low-performing districts and one for 
the rest of the districts, an alternative way to address the same question would have been to include the 
indicator for low-performing district as a covariate as well as an interaction between that variable and 
fidelity. This method is called moderator analysis. The analyst performed two separate analyses instead, 
for ease of communication. 
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A word on “correlation is not causation” 


The analyses discussed in this section, including those in the example scenar- 
ios, are designed to analyze association but not causation. For example, after 
conducting the study described in scenario 3, policymakers might conclude that the type 

of teacher preparation program is associated with where new teachers seek employment, 
which in turn is associated with teacher retention. These results provide policymakers with 
preliminary evidence suggesting that the apparent difference in retention between private 
and public institutions is attributable to a difference in the type of districts in which begin- 
ning teachers seek their first employment rather than to a difference in teacher preparation 
between private and public institutions. Likewise, stakeholders in scenario 1 might conclude 
that the fidelity of implementation of a mentoring program is associated with teacher 
retention. But it would be incorrect for those stakeholders to conclude that implementing 
mentoring programs with higher fidelity causes better teacher retention. Rather, the results 
will hopefully motivate more rigorous studies to investigate whether mentoring programs 
indeed cause increased retention, including in low-performing districts. 


This issue is summarized by the expression “correlation is not causation,” which means that 
just because two variables are correlated does not necessarily mean that one variable is 
causing the other. This is important to remember especially when evaluating programs in 
which mobility or retention is the outcome variable, because stakeholders are often most 
interested in whether the program has the desired impact on mobility or retention. The 
question of whether a program works can be assessed only by studies that permit causal 
interpretations, such as randomized controlled trials, in which participants are randomly 
assigned to conditions, and quasi-experimental designs, in which statistical adjustments and 
matching of participants tries to account for differences between participants who take up a 
program and participants who do not. 


A hypothetical randomized controlled trial for scenario 1 might place individual districts in 
high- versus low-fidelity conditions by randomly withholding state support for program 
implementation. A hypothetical quasi-experimental design for scenario 3 might match 
teachers prepared in private institutions with teachers prepared in public institutions based 
on their background characteristics, creating two groups of teachers that are equivalent in 
terms of background characteristics. 


Because randomized controlled trials and quasi-experimental designs permit comparison of 
two equivalent groups aside from the focal factor—fidelity of implementation—the differ- 
ence in retention between the two groups must be caused by the difference in the fidelity 
of implementation. In contrast, regressions are used to statistically hold constant the differ- 
ences between the two groups in background characteristics, without actually creating two 
groups that are equivalent. The U.S. Department of Education’s What Works Clearinghouse 
(2020) considers randomized controlled trials and quasi-experimental designs capable of 
yielding causal conclusions. 
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Final remarks 


This report—and its companion, report 1—might prove useful to state and district education 
agency teams interested in analyzing teacher mobility and retention. Using the steps and 
example scenarios provided in both reports, multidisciplinary teams and data analysts can 
identify relevant research questions and produce the data to answer them. With this infor- 
mation in hand, stakeholders can inform local policy decisions relating to teacher mobility 
and retention with the goal of creating a stable and equitable workforce for all students. 


As is mentioned many times throughout the report set, this work should not be completed 
solely by a data analyst. It is important to have a multidisciplinary team that includes policy- 
makers, program managers, data managers, and data analysts. When such a team collab- 
orates to co-develop research questions and interpret the findings, the team can identify 
more complete and robust solutions and recommendations. 
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